Clinical Performance of Tumor-Informed versus Tumor-Agnostic ctDNA Assays for Colorectal Cancer Recurrence

A Systematic Review and Diagnostic Accuracy Meta-Analysis

* Main developers of the code.

1 QUADAS-2

QUADAS-2 is a tool for assessing the risk of bias and applicability of diagnostic accuracy studies. Here, we present the results of our QUADAS-2 assessment for the included studies.

quadas %>%
  mutate(Score = factor(Score, levels = c("High", "Unclear", "Low"))) %>%
  count(Domain, Score) %>%
  ggplot(aes(x = n, y = Domain, fill = Score)) +
  geom_bar(stat = "identity", color = "black", linewidth = 0.3) +
  geom_text(aes(label = n), 
            position = position_stack(vjust = 0.5), 
            size = 3.5, color = "black") +
  scale_fill_manual(
    values = c(
      "Low"     = "#A1D99B", 
      "Unclear" = "#FFD92F", 
      "High"    = "#FC9272"
    )
  ) +
  labs(
    x = "Number of Studies",
    y = "Domain",
    fill = "Risk of Bias"
  ) +
  theme(
    axis.title = element_text(size = 13),
    axis.text = element_text(size = 11),
    legend.title = element_text(size = 12),
    legend.text = element_text(size = 10)
  )

ggsave("results/plots/quadas2.pdf", width = 8, height = 5, dpi = 300)
ggsave("results/plots/quadas2.png", width = 8, height = 5, dpi = 300)

2 Number of studies

data %>%
  count(SETTING, APPROACH) %>%
  ggplot(aes(x = SETTING, y = n, fill = APPROACH)) +
  geom_bar(stat = "identity", position = "dodge", color = "black") +
  geom_text(aes(label = n), vjust = 3, position = position_dodge(0.9), color = "white", size = 9) +
  scale_fill_manual(values = colors) +
  labs(x = "Setting", y = "Number of Studies", fill = "Study Type")

There are 44 study results included in the meta-analysis (corresponding to 33 studies; one study can provide results for both the landmark and serial settings), with 27 study results in the landmark setting and 17 study results in the serial setting. The majority of studies are tumor-informed.

3 Descriptive analysis: univariate (DOR) approach

As an initial exploratory analysis, we can use the diagnostic odds ratio (DOR) as a univariate measure of diagnostic accuracy. The DOR is a single summary measure that combines sensitivity and specificity into a single number, which is useful for certain analyses although not ideal for meta-regression, as explained throughout this document.

The diagnostic odds ratio (DOR) can be expressed as:

\[ \text{DOR} = \frac{\text{TP} \times \text{TN}}{\text{FP} \times \text{FN}} \]

3.1 Deeks’ test and funnel plot

The first step of the meta-analysis is to check for publication bias. In the context of diagnostic accuracy studies, this is done using Deeks’ test. This test is based on the idea that if there is no publication bias, the log odds ratio (logOR) should be symmetrically distributed in relation to the Effective Sample Size (ESS).

The ESS quantifies the amount of independent information contained in a sample, especially when the data are correlated. In the context of diagnostic accuracy studies, the ESS can be calculated as:

\[ \text{ESS} = \frac{TP \times TN}{TP + TN + FP + FN} \]

The results from the univariate analysis (based on the DOR) are summarized in the following sections, including the Deeks’ test.

3.1.1 Landmark setting

m_landmark <- metabin(
  data_landmark$TP,
  data_landmark$TP + data_landmark$FP,
  data_landmark$FN,
  data_landmark$FN + data_landmark$TN,
  data = data_landmark,
  sm = "DOR"
)

m_landmark_sg <- metabin(
  data_landmark$TP,
  data_landmark$TP + data_landmark$FP,
  data_landmark$FN,
  data_landmark$FN + data_landmark$TN,
  data = data_landmark,
  subgroup = data_landmark$APPROACH,
  sm = "DOR"
)

summary(m_landmark_sg)

        DOR               95%-CI %W(common) %W(random) APPROACH
1    5.0286 [ 1.7819;   14.1911]        3.0        3.9 Agnostic
2   13.3333 [ 1.6549;  107.4266]        0.5        1.6 Informed
3   15.0000 [ 5.4864;   41.0104]        1.6        4.1 Informed
4   10.0882 [ 2.9176;   34.8824]        2.0        3.3 Informed
5   11.1222 [ 3.6496;   33.8952]        2.0        3.7 Informed
6    3.7500 [ 0.4954;   28.3889]        1.1        1.7 Informed
7   16.4667 [ 4.7178;   57.4744]        1.3        3.2 Informed
8   17.8182 [ 5.4248;   58.5247]        1.5        3.4 Informed
9   24.9885 [13.1008;   47.6631]        3.4        5.7 Informed
10   5.2556 [ 1.7185;   16.0725]        3.0        3.7 Agnostic
11 440.0000 [25.7798; 7509.7454]        0.1        0.9 Agnostic
12  32.7955 [14.5493;   73.9241]        2.1        4.9 Agnostic
13  18.3600 [ 7.5999;   44.3546]        1.9        4.6 Agnostic
14  25.8377 [19.2700;   34.6437]       18.3        7.2 Informed
15  11.3929 [ 3.2913;   39.4365]        2.0        3.3 Agnostic
16  25.6250 [ 5.1244;  128.1390]        0.8        2.3 Agnostic
17  17.2667 [ 3.8331;   77.7794]        0.8        2.6 Informed
18  17.2500 [ 5.3765;   55.3449]        1.4        3.5 Informed
19  10.3333 [ 2.9568;   36.1130]        1.3        3.2 Agnostic
20   6.8148 [ 1.9004;   24.4373]        1.9        3.1 Informed
21  33.9167 [ 8.5595;  134.3934]        0.7        2.9 Informed
22   4.4286 [ 1.5481;   12.6682]        3.5        3.9 Informed
23   3.1081 [ 1.2424;    7.7754]        4.9        4.4 Informed
24  52.2414 [ 2.8129;  970.2336]        0.2        0.9 Informed
25   7.3684 [ 3.6373;   14.9269]        5.3        5.4 Agnostic
26   6.3403 [ 4.5778;    8.7815]       31.0        7.1 Informed
27  14.2759 [ 7.5860;   26.8655]        4.7        5.7 Informed

Number of studies: k = 27
Number of observations: o = 7482 (o.e = 1244, o.c = 6238)
Number of events: e = 1488

                         DOR             95%-CI     z  p-value
Common effect model  13.1534 [11.3389; 15.2584] 34.02 < 0.0001
Random effects model 12.4847 [ 9.3092; 16.7433] 16.86 < 0.0001

Quantifying heterogeneity (with 95%-CIs):
 tau^2 = 0.2881 [0.0891; 0.9325]; tau = 0.5367 [0.2985; 0.9657]
 I^2 = 69.0% [54.0%; 79.1%]; H = 1.80 [1.47; 2.19]

Test of heterogeneity:
     Q d.f.  p-value
 83.95   26 < 0.0001

Results for subgroups (common effect model):
                      k     DOR             95%-CI     Q   I^2
APPROACH = Agnostic   9 13.1091 [ 9.3979; 18.2858] 20.48 60.9%
APPROACH = Informed  18 13.1640 [11.1524; 15.5385] 63.31 73.1%

Test for subgroup differences (common effect model):
                  Q d.f. p-value
Between groups 0.00    1  0.9824

Results for subgroups (random effects model):
                      k     DOR            95%-CI  tau^2    tau
APPROACH = Agnostic   9 13.0253 [7.4817; 22.6767] 0.3709 0.6091
APPROACH = Informed  18 12.2987 [8.5965; 17.5953] 0.2931 0.5414

Test for subgroup differences (random effects model):
                  Q d.f. p-value
Between groups 0.03    1  0.8647

Details of meta-analysis methods:
- Mantel-Haenszel method (common effect model)
- Inverse variance method (random effects model)
- Restricted maximum-likelihood estimator for tau^2
- Q-Profile method for confidence interval of tau^2 and tau
- Calculation of I^2 based on Q
- Continuity correction of 0.5 in studies with zero cell frequencies

The heterogeneity is moderate, which is expected given the different study designs and populations included.

m_landmark_bias <- metabias(m_landmark, method.bias = "Deeks"); m_landmark_bias

Funnel plot test for diagnostic odds ratios

Test result: t = -0.22, df = 25, p-value = 0.8241
Bias estimate: -0.8014 (SE = 3.5683)

Details:
- multiplicative residual heterogeneity variance (tau^2 = 98.0687)
- predictor: inverse of the squared effective sample size
- weight:    effective sample size
- reference: Deeks et al. (2005), J Clin Epid

A p-value > 0.05 indicates that there is no evidence of publication bias.

3.1.2 Serial setting

m_serial <- metabin(
  data_serial$TP,
  data_serial$TP + data_serial$FP,
  data_serial$FN,
  data_serial$FN + data_serial$TN,
  data = data_serial,
  sm = "DOR"
)

m_serial_sg <- metabin(
  data_serial$TP,
  data_serial$TP + data_serial$FP,
  data_serial$FN,
  data_serial$FN + data_serial$TN,
  data = data_serial,
  subgroup = data_serial$APPROACH,
  sm = "DOR"
)

summary(m_serial_sg)

        DOR                95%-CI %W(common) %W(random) APPROACH
1   62.1111 [ 2.9397;  1312.3021]        0.3        3.7 Informed
2   76.0000 [19.5551;   295.3712]        0.9        6.9 Informed
3   27.3913 [ 5.9314;   126.4937]        1.9        6.5 Informed
4  623.0000 [61.6803;  6292.5913]        0.1        4.9 Informed
5   18.0000 [ 1.4957;   216.6201]        0.7        4.6 Agnostic
6   27.4490 [12.6538;    59.5429]        3.9        8.0 Agnostic
7   42.5000 [10.9401;   165.1038]        1.5        6.9 Agnostic
8   45.5556 [ 8.9906;   230.8308]        1.1        6.3 Agnostic
9    3.7125 [ 1.3591;    10.1413]        7.6        7.6 Agnostic
10 143.0000 [ 2.4151;  8467.0059]        0.1        2.6 Informed
11 406.0000 [34.3282;  4801.7697]        0.1        4.7 Informed
12 725.0000 [13.3821; 39278.1701]        0.0        2.7 Informed
13   9.9904 [ 4.0892;    24.4077]        5.7        7.8 Agnostic
14   2.2036 [ 1.4808;     3.2794]       63.9        8.5 Agnostic
15  35.7778 [ 7.9075;   161.8775]        1.2        6.5 Informed
16 273.0000 [13.0822;  5696.9606]        0.1        3.8 Informed
17  10.4942 [ 5.3827;    20.4596]       10.9        8.1 Agnostic

Number of studies: k = 17
Number of observations: o = 2865 (o.e = 590, o.c = 2275)
Number of events: e = 561

                         DOR             95%-CI     z  p-value
Common effect model   9.5138 [ 7.6674; 11.8049] 20.46 < 0.0001
Random effects model 31.4135 [14.3329; 68.8490]  8.61 < 0.0001

Quantifying heterogeneity (with 95%-CIs):
 tau^2 = 1.8542 [0.6926; 5.3405]; tau = 1.3617 [0.8322; 2.3109]
 I^2 = 86.4% [79.7%; 90.9%]; H = 2.71 [2.22; 3.31]

Test of heterogeneity:
      Q d.f.  p-value
 117.69   16 < 0.0001

Results for subgroups (common effect model):
                      k     DOR              95%-CI     Q   I^2
APPROACH = Informed   9 79.6700 [39.6167; 160.2180]  9.56 16.3%
APPROACH = Agnostic   8  6.0253 [ 4.7084;   7.7106] 58.72 88.1%

Test for subgroup differences (common effect model):
                   Q d.f.  p-value
Between groups 46.65    1 < 0.0001

Results for subgroups (random effects model):
                      k     DOR              95%-CI  tau^2    tau
APPROACH = Informed   9 96.5906 [41.1665; 226.6345] 0.4135 0.6430
APPROACH = Agnostic   8 11.4316 [ 5.0729;  25.7607] 1.0378 1.0187

Test for subgroup differences (random effects model):
                   Q d.f. p-value
Between groups 12.61    1  0.0004

Details of meta-analysis methods:
- Mantel-Haenszel method (common effect model)
- Inverse variance method (random effects model)
- Restricted maximum-likelihood estimator for tau^2
- Q-Profile method for confidence interval of tau^2 and tau
- Calculation of I^2 based on Q
- Continuity correction of 0.5 in studies with zero cell frequencies

When evaluating the serial setting, the heterogeneity is lower in tumor-informed studies but increases dramatically in the agnostic studies. This is likely due to the fact that the agnostic studies are more heterogeneous in terms of the methodologies used (genomics, methylomics, and integrated approaches).

m_serial_bias <- metabias(m_serial, method.bias = "Deeks"); m_serial_bias

Funnel plot test for diagnostic odds ratios

Test result: t = 4.10, df = 15, p-value = 0.0009
Bias estimate: 26.7834 (SE = 6.5342)

Details:
- multiplicative residual heterogeneity variance (tau^2 = 151.6305)
- predictor: inverse of the squared effective sample size
- weight:    effective sample size
- reference: Deeks et al. (2005), J Clin Epid

Here, the p-value is < 0.05, suggesting that there is evidence of publication bias. In summary, the Deeks’ test estimate is the slope coefficient from the regression of log DOR on 1/√ESS. Here, a positive estimate indicates that studies with smaller sample sizes (lower ESS) have larger log DORs.

3.1.3 Deeks’ funnel plot

# We generate the funnel plot and extract the data to re-create it with ggplot
pfunnel_landmark <- funnel(m_landmark_sg, yaxis = "ess", studlab = data_landmark$STUDY)

pfunnel_lab_landmark <- paste0(
  "Deeks' test p = ", signif(m_landmark_bias$p.value, 3), "; ",
  "bias estimate = ", signif(m_landmark_bias$estimate[1], 3)
)

pfunnel_landmark <- data.frame(
  x = pfunnel_landmark$xvals, 
  y = pfunnel_landmark$yvals, 
  study = data_landmark$STUDY, 
  approach = data_landmark$APPROACH,
  size = data_landmark$N
  ) %>% 
  ggplot(aes(x = log(x), y = y, text = study, group = approach, color = approach)) +
  geom_line(stat="smooth", method = "lm", size = 0.8, alpha = 0.5) +
  geom_point(aes(size = size)) +
  # geom_text_repel(aes(label = study), size = 3, max.overlaps = 20) +
  scale_color_manual(values = colors) +
  scale_y_reverse() + 
  labs(
    title = "Landmark setting",
    subtitle = pfunnel_lab_landmark,
    x = "Log(Diagnostic Odds Ratio)",
    y = "1 / sqrt(ESS)"
  )

pfunnel_serial <- funnel(m_serial, yaxis = "ess", studlab = data_serial$STUDY)

pfunnel_lab_serial <- paste0(
  "Deeks' test p = ", signif(m_serial_bias$p.value, 3), "; ",
  "bias estimate = ", signif(m_serial_bias$estimate[1], 3)
)

pfunnel_serial <- data.frame(
  x = pfunnel_serial$xvals, 
  y = pfunnel_serial$yvals, 
  study = data_serial$STUDY, 
  approach = data_serial$APPROACH,
  size = data_serial$N
  ) %>% 
  ggplot(aes(x = log(x), y = y, text = study, group = approach, color = approach)) +
  geom_line(stat="smooth", method = "lm", size = 0.8, alpha = 0.5) +
  geom_point(aes(size = size)) +
  # geom_text_repel(aes(label = study), size = 3, max.overlaps = 20) +
  labs(x = "Log(Diagnostic Odds Ratio)", y = "1 / sqrt(ESS)") +
  scale_color_manual(values = colors) +
  scale_y_reverse() + 
  labs(
    title = "Serial setting",
    subtitle = pfunnel_lab_serial,
    x = " ",
    y = ""
  )

pdf("results/plots/pfunnel_landmark_serial.pdf", width = 10, height = 5)
cowplot::plot_grid(
  pfunnel_landmark + theme(legend.position = "none"),
  pfunnel_serial,
  ncol = 2, 
  labels = c("A", "B"), 
  label_size = 12,
  rel_widths = c(0.8, 1)
)
dev.off()

png("results/plots/pfunnel_landmark_serial.png", width = 2600, height = 1200, res = 300)
cowplot::plot_grid(
  pfunnel_landmark + theme(legend.position = "none"),
  pfunnel_serial,
  ncol = 2, 
  labels = c("A", "B"), 
  label_size = 12,
  rel_widths = c(0.8, 1)
)
dev.off()

knitr::include_graphics("results/plots/pfunnel_landmark_serial.png")

3.2 Forest plot

3.2.1 Landmark setting

madad_landmark <- madad(data_landmark %>% mutate(names = STUDY))
madad_landmark

Descriptive summary of data_landmark %>% mutate(names = STUDY) with 27 primary studies.
Confidence level for all calculations set to 95 %
Using a continuity correction of 0.5 if applicable 

Diagnostic accuracies 
                   sens  2.5% 97.5%  spec  2.5% 97.5%
Benhaim_2021      0.283 0.154 0.462 0.927 0.872 0.959
Chan_2022         0.643 0.303 0.882 0.854 0.665 0.945
Chen_2021         0.379 0.234 0.549 0.959 0.923 0.979
Diergaarde_2025   0.763 0.539 0.899 0.739 0.623 0.829
Frydendahl_2024A  0.466 0.299 0.640 0.923 0.845 0.963
Frydendahl_2024B  0.350 0.137 0.646 0.861 0.639 0.956
Gao_2023          0.466 0.299 0.640 0.944 0.872 0.977
Henriksen_2022    0.423 0.282 0.578 0.956 0.898 0.982
Henriksen_2024    0.348 0.270 0.435 0.978 0.965 0.987
Jin_2021          0.548 0.344 0.736 0.806 0.681 0.889
Martín-Arana_2025 0.938 0.769 0.985 0.932 0.751 0.984
Mo_2023           0.775 0.643 0.868 0.900 0.852 0.934
Musher_2020       0.625 0.441 0.779 0.914 0.876 0.941
Nakamura_2024     0.530 0.486 0.574 0.958 0.947 0.966
Nakamura_2024B    0.608 0.448 0.748 0.868 0.716 0.945
Parikh_2021       0.554 0.374 0.720 0.943 0.833 0.982
Reinert_2019      0.417 0.224 0.639 0.955 0.884 0.983
Ryoo_2023         0.630 0.428 0.795 0.903 0.816 0.951
Slater_2024       0.441 0.239 0.665 0.926 0.858 0.963
Tarazona_2019     0.472 0.268 0.687 0.877 0.763 0.941
Tie_2016          0.411 0.250 0.593 0.977 0.939 0.992
Tie_2019          0.420 0.250 0.611 0.856 0.758 0.919
Tie_2025A         0.340 0.187 0.536 0.860 0.813 0.897
Yang_2024         0.341 0.180 0.549 0.990 0.913 0.999
Yuan_2022         0.524 0.376 0.668 0.868 0.825 0.902
Tie_2025B         0.576 0.510 0.638 0.823 0.794 0.849
Cohen_2025        0.553 0.433 0.667 0.919 0.884 0.944

Test for equality of sensitivities: 
X-squared = 79.1578, df = 26, p-value = 2.82e-07
Test for equality of specificities: 
X-squared = 250.6974, df = 26, p-value = <2e-16


Diagnostic OR and likelihood ratios 
                      DOR   2.5%    97.5%  posLR  2.5%   97.5% negLR  2.5%
Benhaim_2021        4.989  1.812   13.734  3.859 1.709   8.710 0.773 0.615
Chan_2022          10.543  1.550   71.703  4.408 1.446  13.438 0.418 0.153
Chen_2021          14.383  5.383   38.429  9.314 4.226  20.527 0.648 0.495
Diergaarde_2025     9.114  2.773   29.954  2.922 1.818   4.695 0.321 0.141
Frydendahl_2024A   10.385  3.516   30.672  6.016 2.610  13.867 0.579 0.410
Frydendahl_2024B    3.338  0.518   21.523  2.520 0.605  10.500 0.755 0.462
Gao_2023           14.806  4.467   49.083  8.379 3.148  22.303 0.566 0.401
Henriksen_2022     16.052  5.141   50.116  9.684 3.653  25.674 0.603 0.460
Henriksen_2024     24.276 12.835   45.917 16.176 9.214  28.400 0.666 0.586
Jin_2021            5.015  1.683   14.947  2.816 1.444   5.491 0.562 0.345
Martín-Arana_2025 205.000 19.728 2130.184 13.750 2.924  64.668 0.067 0.014
Mo_2023            31.081 13.971   69.145  7.783 5.029  12.044 0.250 0.150
Musher_2020        17.680  7.436   42.038  7.255 4.539  11.596 0.410 0.254
Nakamura_2024      25.663 19.154   34.385 12.587 9.916  15.979 0.490 0.447
Nakamura_2024B     10.172  3.090   33.486  4.595 1.871  11.286 0.452 0.296
Parikh_2021        20.584  4.695   90.238  9.743 2.794  33.971 0.473 0.311
Reinert_2019       15.204  3.661   63.140  9.286 2.909  29.640 0.611 0.412
Ryoo_2023          15.808  5.090   49.097  6.472 3.062  13.681 0.409 0.239
Slater_2024         9.842  2.928   33.084  5.941 2.484  14.209 0.604 0.394
Tarazona_2019       6.401  1.858   22.050  3.850 1.613   9.191 0.602 0.384
Tie_2016           29.571  8.058  108.518 17.837 5.782  55.027 0.603 0.442
Tie_2019            4.310  1.541   12.057  2.920 1.414   6.028 0.677 0.479
Tie_2025A           3.166  1.292    7.762  2.430 1.305   4.524 0.767 0.577
Yang_2024          52.241  2.813  970.234 34.773 2.075 582.630 0.666 0.492
Yuan_2022           7.271  3.617   14.618  3.983 2.639   6.011 0.548 0.396
Tie_2025B           6.313  4.561    8.737  3.255 2.687   3.943 0.516 0.441
Cohen_2025         13.984  7.469   26.180  6.803 4.450  10.401 0.487 0.371
                  97.5%
Benhaim_2021      0.973
Chan_2022         1.145
Chen_2021         0.846
Diergaarde_2025   0.728
Frydendahl_2024A  0.818
Frydendahl_2024B  1.234
Gao_2023          0.798
Henriksen_2022    0.792
Henriksen_2024    0.758
Jin_2021          0.915
Martín-Arana_2025 0.317
Mo_2023           0.417
Musher_2020       0.663
Nakamura_2024     0.539
Nakamura_2024B    0.689
Parikh_2021       0.720
Reinert_2019      0.905
Ryoo_2023         0.702
Slater_2024       0.924
Tarazona_2019     0.942
Tie_2016          0.823
Tie_2019          0.958
Tie_2025A         1.021
Yang_2024         0.900
Yuan_2022         0.758
Tie_2025B         0.603
Cohen_2025        0.638

Correlation of sensitivities and false positive rates: 
   rho  2.5 % 97.5 % 
 0.369 -0.013  0.657 

meta::forest(
  m_landmark_sg, 
  studlab = data_landmark$STUDY, 
  col.diamond = "black", 
  file = "results/plots/pforest_landmark.pdf",
  width = 12
  )

meta::forest(
  m_landmark_sg, 
  studlab = data_landmark$STUDY, 
  col.diamond = "black", 
  file = "results/plots/pforest_landmark.png",
  width = 900
  )

knitr::include_graphics("results/plots/pforest_landmark.png")

3.2.2 Serial setting

madad_serial <- madad(data_serial %>% mutate(names = STUDY))
madad_serial

Descriptive summary of data_serial %>% mutate(names = STUDY) with 17 primary studies.
Confidence level for all calculations set to 95 %
Using a continuity correction of 0.5 if applicable 

Diagnostic accuracies 
                 sens  2.5% 97.5%  spec  2.5% 97.5%
Chan_2022       0.929 0.561 0.992 0.827 0.643 0.927
Chen_2021       0.812 0.618 0.921 0.937 0.872 0.970
Diergaarde_2025 0.891 0.705 0.966 0.730 0.628 0.812
Henriksen_2022  0.860 0.677 0.947 0.984 0.932 0.996
Jin_2021        0.812 0.467 0.955 0.731 0.460 0.896
Nakamura_2024A  0.678 0.532 0.796 0.926 0.890 0.951
Nakamura_2024B  0.824 0.673 0.914 0.885 0.748 0.952
Parikh_2021     0.683 0.504 0.821 0.943 0.833 0.982
Pedersen_2023   0.279 0.157 0.447 0.905 0.835 0.947
Reinert_2016    0.929 0.561 0.992 0.917 0.517 0.991
Reinert_2019    0.853 0.623 0.953 0.975 0.899 0.994
Schøler_2017    0.967 0.747 0.997 0.962 0.717 0.996
Slater_2024     0.617 0.439 0.768 0.857 0.788 0.906
Taieb_2021      0.227 0.174 0.291 0.883 0.859 0.903
Tarazona_2019   0.806 0.577 0.926 0.877 0.763 0.941
Wang_2019       0.955 0.679 0.995 0.929 0.821 0.974
Yuan_2022       0.716 0.580 0.821 0.802 0.759 0.839

Test for equality of sensitivities: 
X-squared = 182.3157, df = 16, p-value = <2e-16
Test for equality of specificities: 
X-squared = 71.2468, df = 16, p-value = 6.03e-09


Diagnostic OR and likelihood ratios 
                    DOR   2.5%     97.5%  posLR   2.5%   97.5% negLR  2.5%
Chan_2022        62.111  2.940  1312.302  5.365  2.259  12.741 0.086 0.006
Chen_2021        64.333 17.591   235.283 12.875  5.970  27.766 0.200 0.087
Diergaarde_2025  22.157  5.488    89.466  3.300  2.271   4.795 0.149 0.046
Henriksen_2022  366.524 51.124  2627.707 52.173 10.587 257.107 0.142 0.054
Jin_2021         11.762  1.360   101.713  3.018  1.161   7.846 0.257 0.058
Nakamura_2024A   26.366 12.281    56.609  9.174  5.826  14.444 0.348 0.227
Nakamura_2024B   35.974  9.835   131.592  7.144  2.958  17.252 0.199 0.098
Parikh_2021      35.821  8.071   158.984 12.027  3.521  41.074 0.336 0.197
Pedersen_2023     3.674  1.375     9.816  2.927  1.330   6.442 0.797 0.641
Reinert_2016    143.000  2.415  8467.006 11.143  0.778 159.580 0.078 0.005
Reinert_2019    226.200 27.580  1855.179 34.118  6.940 167.715 0.151 0.048
Schøler_2017    725.000 13.382 39278.170 25.133  1.656 381.381 0.035 0.002
Slater_2024       9.611  3.993    23.135  4.301  2.613   7.079 0.447 0.283
Taieb_2021        2.211  1.488     3.286  1.936  1.407   2.664 0.876 0.808
Tarazona_2019    29.637  7.114   123.464  6.568  3.087  13.975 0.222 0.086
Wang_2019       273.000 13.082  5696.961 13.364  4.830  36.976 0.049 0.003
Yuan_2022        10.221  5.287    19.758  3.622  2.773   4.730 0.354 0.229
                97.5%
Chan_2022       1.256
Chen_2021       0.461
Diergaarde_2025 0.483
Henriksen_2022  0.376
Jin_2021        1.127
Nakamura_2024A  0.532
Nakamura_2024B  0.403
Parikh_2021     0.571
Pedersen_2023   0.991
Reinert_2016    1.139
Reinert_2019    0.474
Schøler_2017    0.530
Slater_2024     0.708
Taieb_2021      0.948
Tarazona_2019   0.571
Wang_2019       0.735
Yuan_2022       0.549

Correlation of sensitivities and false positive rates: 
   rho  2.5 % 97.5 % 
-0.014 -0.492  0.470 

meta::forest(
  m_serial_sg, 
  studlab = data_serial$STUDY, 
  col.diamond = "black", 
  file = "results/plots/pforest_serial.pdf",
  width = 12
  )

meta::forest(
  m_serial_sg, 
  studlab = data_serial$STUDY, 
  col.diamond = "black", 
  file = "results/plots/pforest_serial.png",
  width = 900
  )

knitr::include_graphics("results/plots/pforest_serial.png")

4 Descriptive analysis: pooled sensitivity, specificity, PPV and NPV

We can also perform a univariate meta-analysis of sensitivity and specificity separately, using a random-effects model. This approach does not account for the correlation between sensitivity and specificity, which is a limitation. However, it provides a useful summary of the data.

In this section, we present, in order, the pooled estimates of sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for both the landmark and serial settings.

4.1 Landmark setting

# Sensitivity
m_land_sens <- run_diagnostic_prop(
  event = data_landmark$TP, 
  n = data_landmark$TP + data_landmark$FN, 
  data = data_landmark, 
  label = "Sensitivity", 
  APPROACH = data_landmark$APPROACH
)

# Specificity
m_land_spec <- run_diagnostic_prop(
  event = data_landmark$TN, 
  n = data_landmark$TN + data_landmark$FP, 
  data = data_landmark, 
  label = "Specificity", 
  APPROACH = data_landmark$APPROACH
)

# PPV
m_land_ppv <- run_diagnostic_prop(
  event = data_landmark$TP, 
  n = data_landmark$TP + data_landmark$FP, 
  data = data_landmark, 
  label = "PPV", 
  APPROACH = data_landmark$APPROACH
)

# NPV
m_land_npv <- run_diagnostic_prop(
  event = data_landmark$TN, 
  n = data_landmark$TN + data_landmark$FN, 
  data = data_landmark, 
  label = "NPV", 
  APPROACH = data_landmark$APPROACH
)

save_forest(m_land_sens, data_landmark$STUDY, "pforest_landmark_sens")
save_forest(m_land_spec, data_landmark$STUDY, "pforest_landmark_spec")
save_forest(m_land_ppv,  data_landmark$STUDY, "pforest_landmark_ppv")
save_forest(m_land_npv,  data_landmark$STUDY, "pforest_landmark_npv")

knitr::include_graphics(c(
  "results/plots/pforest_landmark_sens.png",
  "results/plots/pforest_landmark_spec.png",
  "results/plots/pforest_landmark_ppv.png",
  "results/plots/pforest_landmark_npv.png"
))

4.2 Serial setting

# Sensitivity
m_serial_sens <- run_diagnostic_prop(
  event = data_serial$TP, 
  n = data_serial$TP + data_serial$FN, 
  data = data_serial, 
  label = "Sensitivity", 
  APPROACH = data_serial$APPROACH
)

# Specificity
m_serial_spec <- run_diagnostic_prop(
  event = data_serial$TN, 
  n = data_serial$TN + data_serial$FP, 
  data = data_serial, 
  label = "Specificity", 
  APPROACH = data_serial$APPROACH
)

# PPV
m_serial_ppv <- run_diagnostic_prop(
  event = data_serial$TP, 
  n = data_serial$TP + data_serial$FP, 
  data = data_serial, 
  label = "PPV", 
  APPROACH = data_serial$APPROACH
)

# NPV
m_serial_npv <- run_diagnostic_prop(
  event = data_serial$TN, 
  n = data_serial$TN + data_serial$FN, 
  data = data_serial, 
  label = "NPV", 
  APPROACH = data_serial$APPROACH
)

save_forest(m_serial_sens, data_serial$STUDY, "pforest_serial_sens")
save_forest(m_serial_spec, data_serial$STUDY, "pforest_serial_spec")
save_forest(m_serial_ppv,  data_serial$STUDY, "pforest_serial_ppv")
save_forest(m_serial_npv,  data_serial$STUDY, "pforest_serial_npv")

knitr::include_graphics(c(
  "results/plots/pforest_serial_sens.png",
  "results/plots/pforest_serial_spec.png",
  "results/plots/pforest_serial_ppv.png",
  "results/plots/pforest_serial_npv.png"
))

5 Bivariate Meta-Analysis

As stated before, bivariate meta-analysis according to the Reitsma model combines the sensivitiy and specificity. The model assumes that the true positive rate (TPR) and false positive rate (FPR) are correlated.

5.1 Landmark vs. Serial

We can perform a initial bivariate meta-analysis comparison of the landmark and serial data. It is expected that the serial analysis of ctDNA yields a more accurate assessment of the risk of recurrence than the landmark analysis.

fit_landmark <- reitsma(data_landmark)
fit_serial <- reitsma(data_serial)

png("results/plots/sroc_global.png", width = 550, height = 550, res = 120)

# Extract the data for the Landmark and Serial SROC curves
landmark_sroc <- sroc(fit_landmark)
serial_sroc <- sroc(fit_serial)

# Start with a blank plot for the SROC curves
plot(
  landmark_sroc, 
  xlim = c(0, 0.3), 
  ylim = c(0, 1), 
  main = "Landmark vs. Serial", 
  xlab = "False Positive Rate",
  ylab = "Sensitivity",
  type = "n"  # 'type = "n"' makes a blank plot
  )  

# Add the SROC curves
lines(landmark_sroc, col = "#3E6D9C", lwd = 2)
lines(serial_sroc, col = "#FD841F", lwd = 2)

# Add confidence ellipses
ROCellipse(fit_landmark, pch = 1, col = "#3E6D9C", add = TRUE)  # Landmark ellipses
ROCellipse(fit_serial, pch = 2, col = "#FD841F", add = TRUE)  # Serial ellipses

# Add points
points(fpr(data_landmark), sens(data_landmark), col = "#3E6D9C", cex = 0.5)
points(fpr(data_serial), sens(data_serial), col = "#FD841F", pch = 2, cex = 0.5)

# Add a legend
legend("bottomright", legend = c("Landmark", "Serial"), col = c("#3E6D9C", "#FD841F"), pch = 1:2, lty = 1:2)

dev.off()

knitr::include_graphics("results/plots/sroc_global.png")

fit_all <- reitsma(data, formula = cbind(tsens, tfpr) ~ SETTING)
summary(fit_all)

Call:  reitsma.default(data = data, formula = cbind(tsens, tfpr) ~ SETTING)

Bivariate diagnostic random-effects meta-analysis
Estimation method: REML

Fixed-effects coefficients
                    Estimate Std. Error       z Pr(>|z|) 95%ci.lb 95%ci.ub    
tsens.(Intercept)      0.048      0.154   0.311    0.756   -0.254    0.350    
tsens.SETTINGSerial    0.888      0.271   3.276    0.001    0.357    1.419  **
tfpr.(Intercept)      -2.369      0.138 -17.177    0.000   -2.639   -2.098 ***
tfpr.SETTINGSerial     0.313      0.232   1.350    0.177   -0.141    0.767    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Variance components: between-studies Std. Dev and correlation matrix
      Std. Dev tsens  tfpr
tsens    0.689 1.000     .
tfpr     0.597 0.279 1.000

  logLik      AIC      BIC 
  82.546 -151.091 -133.750 


I2 estimates 
Zhou and Dendukuri approach:  20.4 % 
Holling sample size unadjusted approaches:  42.3 - 81.1 % 
Holling sample size adjusted approaches:  3.6 - 5 %

To have a clearer picture of the results, we can transform logit estimates into percentages.

• The intercept represents the base value of the reference category.
• The logit estimate below corresponds to the (logit) difference

summary(fit_all)$coefficients %>% 
  as.data.frame() %>% 
  rownames_to_column("Var") %>%
  select(Var, Estimate, `95%ci.lb`, `95%ci.ub`) %>%
  mutate(
    Measure = case_when(
      str_detect(Var, "^tsens") ~ "Sensitivity",
      str_detect(Var, "^tfpr") ~ "False Positive Rate",
      TRUE ~ NA_character_
    ),
    Type = case_when(
      str_detect(Var, "Intercept") ~ "Intercept",
      str_detect(Var, "Serial") ~ "Effect",
      TRUE ~ NA_character_
    )
  ) %>%
  # Spread intercepts to join with effects
  group_by(Measure) %>%
  mutate(
    Intercept_Estimate = Estimate[Type == "Intercept"],
    Intercept_LB = `95%ci.lb`[Type == "Intercept"],
    Intercept_UB = `95%ci.ub`[Type == "Intercept"]
  ) %>%
  ungroup() %>%
  # Calculate combined logit for effects, or keep intercept
  mutate(
    Logit = if_else(Type == "Effect", Estimate + Intercept_Estimate, Estimate),
    Logit_LB = if_else(Type == "Effect", `95%ci.lb` + Intercept_LB, `95%ci.lb`),
    Logit_UB = if_else(Type == "Effect", `95%ci.ub` + Intercept_UB, `95%ci.ub`),
    Probability = plogis(Logit),
    Probability_LB = plogis(Logit_LB),
    Probability_UB = plogis(Logit_UB)
  ) %>%
  select(Measure, Type, Probability, Probability_LB, Probability_UB) %>% 
  mutate(
    Type = case_when(
      Type == "Intercept" ~ "Landmark",
      Type == "Effect" ~ "Serial"
    )
  ) %>% 
  knitr::kable()

Measure	Type	Probability	Probability_LB	Probability_UB
Sensitivity	Landmark	0.5119845	0.4368186	0.5866121
Sensitivity	Serial	0.7182504	0.5256298	0.8543319
False Positive Rate	Landmark	0.0856028	0.0666826	0.1092629
False Positive Rate	Serial	0.1134488	0.0584042	0.2088641

Using a serial approach significantly increases the sensitivity of recurrence detection compared to a landmark approach. The true-false positive rate raises slightly, but the increase is non-significant.

5.2 Tumor-Informed vs. Tumor-Agnostic

data_landmark_inf <- data_landmark %>% filter(APPROACH == "Informed")
data_landmark_agn <- data_landmark %>% filter(APPROACH == "Agnostic")

data_serial_inf <- data_serial %>% filter(APPROACH == "Informed")
data_serial_agn <- data_serial %>% filter(APPROACH == "Agnostic")

5.2.0.1 Landmark setting

fit_landmark_inf <- reitsma(data_landmark_inf)
fit_landmark_agn <- reitsma(data_landmark_agn)

5.2.0.1.1 SROC curves

png("results/plots/sroc_landmark.png", width = 610, height = 610, res = 140)

# Extract the data for the Informed and Agnostic SROC curves
landmark_inf_sroc <- sroc(fit_landmark_inf)
landmark_agn_sroc <- sroc(fit_landmark_agn)

# Start with a blank plot for the SROC curves
plot(
  landmark_inf_sroc, 
  xlim = c(0, 0.3), 
  ylim = c(0, 1), 
  main = "Landmark setting", 
  xlab = "False Positive Rate",
  ylab = "Sensitivity",
  type = "n"
  )

# Add the SROC curves
lines(landmark_inf_sroc, col = colors["Informed"], lwd = 2)
lines(landmark_agn_sroc, col = colors["Agnostic"], lwd = 2)

# Add confidence ellipses
ROCellipse(fit_landmark_inf, pch = 1, col = colors["Informed"], add = TRUE)  # Landmark ellipses
ROCellipse(fit_landmark_agn, pch = 2, col = colors["Agnostic"], add = TRUE)  # Serial ellipses

# Add points
points(fpr(data_landmark_inf), sens(data_landmark_inf), col = colors["Informed"], cex = 0.5)
points(fpr(data_landmark_agn), sens(data_landmark_agn), col = colors["Agnostic"], pch = 2, cex = 0.5)

# Add a legend
legend("bottomright", legend = c("Informed", "Agnostic"), col = c(colors["Informed"], colors["Agnostic"]), pch = 1:2, lty = 1:2)

dev.off()

knitr::include_graphics("results/plots/sroc_landmark.png")

5.2.0.1.2 Bivariate modeling

fit_landmark <- reitsma(data_landmark, formula = cbind(tsens, tfpr) ~ APPROACH)
summary(fit_landmark)

Call:  reitsma.default(data = data_landmark, formula = cbind(tsens, 
    tfpr) ~ APPROACH)

Bivariate diagnostic random-effects meta-analysis
Estimation method: REML

Fixed-effects coefficients
                       Estimate Std. Error      z Pr(>|z|) 95%ci.lb 95%ci.ub
tsens.(Intercept)         0.322      0.182  1.769    0.077   -0.035    0.679
tsens.APPROACHInformed   -0.395      0.218 -1.813    0.070   -0.822    0.032
tfpr.(Intercept)         -2.181      0.250 -8.710    0.000   -2.672   -1.690
tfpr.APPROACHInformed    -0.300      0.307 -0.977    0.329   -0.903    0.302
                          
tsens.(Intercept)        .
tsens.APPROACHInformed   .
tfpr.(Intercept)       ***
tfpr.APPROACHInformed     
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Variance components: between-studies Std. Dev and correlation matrix
      Std. Dev tsens  tfpr
tsens    0.364 1.000     .
tfpr     0.641 0.605 1.000

  logLik      AIC      BIC 
  58.461 -102.921  -88.998 


I2 estimates 
Zhou and Dendukuri approach:  21.4 % 
Holling sample size unadjusted approaches:  30.5 - 55.4 % 
Holling sample size adjusted approaches:  1.3 - 1.4 %

summary(fit_landmark)$coefficients %>% 
  as.data.frame() %>% 
  rownames_to_column("Var") %>%
  select(Var, Estimate, `95%ci.lb`, `95%ci.ub`) %>%
  mutate(
    Measure = case_when(
      str_detect(Var, "^tsens") ~ "Sensitivity",
      str_detect(Var, "^tfpr") ~ "False Positive Rate",
      TRUE ~ NA_character_
    ),
    Type = case_when(
      str_detect(Var, "Intercept") ~ "Intercept",
      str_detect(Var, "Informed") ~ "Effect",
      TRUE ~ NA_character_
    )
  ) %>%
  # Spread intercepts to join with effects
  group_by(Measure) %>%
  mutate(
    Intercept_Estimate = Estimate[Type == "Intercept"],
    Intercept_LB = `95%ci.lb`[Type == "Intercept"],
    Intercept_UB = `95%ci.ub`[Type == "Intercept"]
  ) %>%
  ungroup() %>%
  # Calculate combined logit for effects, or keep intercept
  mutate(
    Logit = if_else(Type == "Effect", Estimate + Intercept_Estimate, Estimate),
    Logit_LB = if_else(Type == "Effect", `95%ci.lb` + Intercept_LB, `95%ci.lb`),
    Logit_UB = if_else(Type == "Effect", `95%ci.ub` + Intercept_UB, `95%ci.ub`),
    Probability = plogis(Logit),
    Probability_LB = plogis(Logit_LB),
    Probability_UB = plogis(Logit_UB)
  ) %>%
  select(Measure, Type, Probability, Probability_LB, Probability_UB) %>% 
  mutate(
    Type = case_when(
      Type == "Intercept" ~ "Agnostic",
      Type == "Effect" ~ "Informed"
    )
  ) %>% 
  knitr::kable()

Measure	Type	Probability	Probability_LB	Probability_UB
Sensitivity	Agnostic	0.5798337	0.4913178	0.6634962
Sensitivity	Informed	0.4817765	0.2979967	0.6706228
False Positive Rate	Agnostic	0.1014570	0.0646497	0.1557310
False Positive Rate	Informed	0.0771706	0.0272632	0.1996835

5.2.0.2 Serial setting

fit_serial_inf <- reitsma(data_serial_inf)
fit_serial_agn <- reitsma(data_serial_agn)

5.2.0.2.1 SROC curves

png("results/plots/sroc_serial.png", width = 610, height = 610, res = 140)

# Extract the data for the Informed and Agnostic SROC curves
serial_inf_sroc <- sroc(fit_serial_inf)
serial_agn_sroc <- sroc(fit_serial_agn)

# Start with a blank plot for the SROC curves
plot(
  serial_inf_sroc, 
  xlim = c(0, 0.3), 
  ylim = c(0, 1), 
  main = "Serial setting", 
  xlab = "False Positive Rate",
  ylab = "Sensitivity",
  type = "n"  # 'type = "n"' makes a blank plot
  )

# Add the SROC curves
lines(serial_inf_sroc, col = colors["Informed"], lwd = 2)
lines(serial_agn_sroc, col = colors["Agnostic"], lwd = 2)

# Add confidence ellipses
ROCellipse(fit_serial_inf, pch = 1, col = colors["Informed"], add = TRUE)
ROCellipse(fit_serial_agn, pch = 2, col = colors["Agnostic"], add = TRUE)

# Add points
points(fpr(data_serial_inf), sens(data_serial_inf), col = colors["Informed"], cex = 0.5)
points(fpr(data_serial_agn), sens(data_serial_agn), col = colors["Agnostic"], pch = 2, cex = 0.5)

# Add a legend
legend("bottomright", legend = c("Informed", "Agnostic"), col = c(colors["Informed"], colors["Agnostic"]), pch = 1:2, lty = 1:2)

dev.off()

knitr::include_graphics("results/plots/sroc_serial.png")

5.2.0.2.2 Bivariate modeling

fit_serial <- reitsma(data_serial, formula = cbind(tsens, tfpr) ~ APPROACH)
summary(fit_serial)

Call:  reitsma.default(data = data_serial, formula = cbind(tsens, tfpr) ~ 
    APPROACH)

Bivariate diagnostic random-effects meta-analysis
Estimation method: REML

Fixed-effects coefficients
                       Estimate Std. Error      z Pr(>|z|) 95%ci.lb 95%ci.ub
tsens.(Intercept)         0.373      0.311  1.199    0.231   -0.237    0.984
tsens.APPROACHInformed    1.598      0.499  3.202    0.001    0.620    2.576
tfpr.(Intercept)         -1.954      0.241 -8.090    0.000   -2.427   -1.480
tfpr.APPROACHInformed    -0.302      0.374 -0.808    0.419   -1.034    0.430
                          
tsens.(Intercept)         
tsens.APPROACHInformed  **
tfpr.(Intercept)       ***
tfpr.APPROACHInformed     
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Variance components: between-studies Std. Dev and correlation matrix
      Std. Dev tsens  tfpr
tsens    0.781 1.000     .
tfpr     0.588 0.225 1.000

 logLik     AIC     BIC 
 35.538 -57.076 -46.391 


I2 estimates 
Zhou and Dendukuri approach:  37.4 % 
Holling sample size unadjusted approaches:  59.1 - 91.5 % 
Holling sample size adjusted approaches:  9 - 12.8 %

summary(fit_serial)$coefficients %>% 
  as.data.frame() %>% 
  rownames_to_column("Var") %>%
  select(Var, Estimate, `95%ci.lb`, `95%ci.ub`) %>%
  mutate(
    Measure = case_when(
      str_detect(Var, "^tsens") ~ "Sensitivity",
      str_detect(Var, "^tfpr") ~ "False Positive Rate",
      TRUE ~ NA_character_
    ),
    Type = case_when(
      str_detect(Var, "Intercept") ~ "Intercept",
      str_detect(Var, "Informed") ~ "Effect",
      TRUE ~ NA_character_
    )
  ) %>%
  # Spread intercepts to join with effects
  group_by(Measure) %>%
  mutate(
    Intercept_Estimate = Estimate[Type == "Intercept"],
    Intercept_LB = `95%ci.lb`[Type == "Intercept"],
    Intercept_UB = `95%ci.ub`[Type == "Intercept"]
  ) %>%
  ungroup() %>%
  # Calculate combined logit for effects, or keep intercept
  mutate(
    Logit = if_else(Type == "Effect", Estimate + Intercept_Estimate, Estimate),
    Logit_LB = if_else(Type == "Effect", `95%ci.lb` + Intercept_LB, `95%ci.lb`),
    Logit_UB = if_else(Type == "Effect", `95%ci.ub` + Intercept_UB, `95%ci.ub`),
    Probability = plogis(Logit),
    Probability_LB = plogis(Logit_LB),
    Probability_UB = plogis(Logit_UB)
  ) %>%
  select(Measure, Type, Probability, Probability_LB, Probability_UB) %>% 
  mutate(
    Type = case_when(
      Type == "Intercept" ~ "Agnostic",
      Type == "Effect" ~ "Informed"
    )
  ) %>% 
  knitr::kable()

Measure	Type	Probability	Probability_LB	Probability_UB
Sensitivity	Agnostic	0.5922713	0.4409883	0.7278769
Sensitivity	Informed	0.8777684	0.5945644	0.9723493
False Positive Rate	Agnostic	0.1241449	0.0811312	0.1853631
False Positive Rate	Informed	0.0948629	0.0304364	0.2592059

5.3 Effect of methodology and assay type

5.3.1 Landmark setting

reitsma(
  data_landmark %>%
    filter(APPROACH == "Informed") %>%
    mutate(METHODOLOGY_CLEAN = factor(METHODOLOGY_CLEAN, levels = c("Panel", "Genome-wide"))),
  formula = cbind(tsens, tfpr) ~ METHODOLOGY_CLEAN
  ) %>%
  summary()

Call:  reitsma.default(data = data_landmark %>% filter(APPROACH == "Informed") %>% 
    mutate(METHODOLOGY_CLEAN = factor(METHODOLOGY_CLEAN, levels = c("Panel", 
        "Genome-wide"))), formula = cbind(tsens, tfpr) ~ METHODOLOGY_CLEAN)

Bivariate diagnostic random-effects meta-analysis
Estimation method: REML

Fixed-effects coefficients
                                   Estimate Std. Error      z Pr(>|z|) 95%ci.lb
tsens.(Intercept)                    -0.333      0.176 -1.896    0.058   -0.677
tsens.METHODOLOGY_CLEANGenome-wide    0.418      0.215  1.946    0.052   -0.003
tfpr.(Intercept)                     -2.425      0.311 -7.798    0.000   -3.034
tfpr.METHODOLOGY_CLEANGenome-wide    -0.106      0.420 -0.253    0.800   -0.928
                                   95%ci.ub    
tsens.(Intercept)                     0.011   .
tsens.METHODOLOGY_CLEANGenome-wide    0.840   .
tfpr.(Intercept)                     -1.815 ***
tfpr.METHODOLOGY_CLEANGenome-wide     0.716    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Variance components: between-studies Std. Dev and correlation matrix
      Std. Dev tsens  tfpr
tsens    0.282 1.000     .
tfpr     0.783 0.871 1.000

 logLik     AIC     BIC 
 44.542 -75.085 -64.000 


I2 estimates 
Zhou and Dendukuri approach:  13.4 % 
Holling sample size unadjusted approaches:  23.2 - 48.4 % 
Holling sample size adjusted approaches:  0.9 - 0.9 %

reitsma(
  data_landmark %>%
    filter(APPROACH == "Agnostic") %>%
    mutate(METHODOLOGY_CLEAN = factor(METHODOLOGY_CLEAN, levels = c("Single-omic", "Multi-omic"))),
  formula = cbind(tsens, tfpr) ~ METHODOLOGY_CLEAN
  ) %>%
  summary()

Call:  reitsma.default(data = data_landmark %>% filter(APPROACH == "Agnostic") %>% 
    mutate(METHODOLOGY_CLEAN = factor(METHODOLOGY_CLEAN, levels = c("Single-omic", 
        "Multi-omic"))), formula = cbind(tsens, tfpr) ~ METHODOLOGY_CLEAN)

Bivariate diagnostic random-effects meta-analysis
Estimation method: REML

Fixed-effects coefficients
                                  Estimate Std. Error       z Pr(>|z|) 95%ci.lb
tsens.(Intercept)                    0.466      0.352   1.323    0.186   -0.224
tsens.METHODOLOGY_CLEANMulti-omic   -0.302      0.596  -0.507    0.612   -1.469
tfpr.(Intercept)                    -2.136      0.152 -14.033    0.000   -2.435
tfpr.METHODOLOGY_CLEANMulti-omic    -0.339      0.360  -0.942    0.346   -1.044
                                  95%ci.ub    
tsens.(Intercept)                    1.155    
tsens.METHODOLOGY_CLEANMulti-omic    0.866    
tfpr.(Intercept)                    -1.838 ***
tfpr.METHODOLOGY_CLEANMulti-omic     0.366    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Variance components: between-studies Std. Dev and correlation matrix
      Std. Dev tsens  tfpr
tsens    0.724 1.000     .
tfpr     0.238 0.106 1.000

 logLik     AIC     BIC 
 20.405 -26.809 -20.577 


I2 estimates 
Zhou and Dendukuri approach:  65.1 % 
Holling sample size unadjusted approaches:  51.7 - 68 % 
Holling sample size adjusted approaches:  3.4 - 4.9 %

5.3.2 Serial setting

reitsma(
  data_serial %>%
    filter(APPROACH == "Informed") %>%
    mutate(METHODOLOGY_CLEAN = factor(METHODOLOGY_CLEAN, levels = c("Panel", "Genome-wide"))),
  formula = cbind(tsens, tfpr) ~ METHODOLOGY_CLEAN
  ) %>%
  summary()

Call:  reitsma.default(data = data_serial %>% filter(APPROACH == "Informed") %>% 
    mutate(METHODOLOGY_CLEAN = factor(METHODOLOGY_CLEAN, levels = c("Panel", 
        "Genome-wide"))), formula = cbind(tsens, tfpr) ~ METHODOLOGY_CLEAN)

Bivariate diagnostic random-effects meta-analysis
Estimation method: REML

Fixed-effects coefficients
                                   Estimate Std. Error      z Pr(>|z|) 95%ci.lb
tsens.(Intercept)                     1.633      0.371  4.401    0.000    0.906
tsens.METHODOLOGY_CLEANGenome-wide    0.383      0.510  0.751    0.452   -0.617
tfpr.(Intercept)                     -2.200      0.529 -4.156    0.000   -3.237
tfpr.METHODOLOGY_CLEANGenome-wide    -0.415      0.784 -0.529    0.597   -1.952
                                   95%ci.ub    
tsens.(Intercept)                     2.360 ***
tsens.METHODOLOGY_CLEANGenome-wide    1.384    
tfpr.(Intercept)                     -1.163 ***
tfpr.METHODOLOGY_CLEANGenome-wide     1.122    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Variance components: between-studies Std. Dev and correlation matrix
      Std. Dev tsens  tfpr
tsens    0.102 1.000     .
tfpr     0.946 1.000 1.000

 logLik     AIC     BIC 
 26.190 -38.379 -32.147 


I2 estimates 
Zhou and Dendukuri approach:  0 % 
Holling sample size unadjusted approaches:  0 - 0 % 
Holling sample size adjusted approaches:  0 - 0 %

reitsma(
  data_serial %>%
    filter(APPROACH == "Agnostic") %>%
    mutate(METHODOLOGY_CLEAN = factor(METHODOLOGY_CLEAN, levels = c("Single-omic", "Multi-omic"))),
  formula = cbind(tsens, tfpr) ~ METHODOLOGY_CLEAN
  ) %>%
  summary()

Call:  reitsma.default(data = data_serial %>% filter(APPROACH == "Agnostic") %>% 
    mutate(METHODOLOGY_CLEAN = factor(METHODOLOGY_CLEAN, levels = c("Single-omic", 
        "Multi-omic"))), formula = cbind(tsens, tfpr) ~ METHODOLOGY_CLEAN)

Bivariate diagnostic random-effects meta-analysis
Estimation method: REML

Fixed-effects coefficients
                                  Estimate Std. Error      z Pr(>|z|) 95%ci.lb
tsens.(Intercept)                   -0.133      0.502 -0.265    0.791   -1.117
tsens.METHODOLOGY_CLEANMulti-omic    1.047      0.698  1.499    0.134   -0.322
tfpr.(Intercept)                    -1.726      0.189 -9.113    0.000   -2.097
tfpr.METHODOLOGY_CLEANMulti-omic    -0.480      0.296 -1.625    0.104   -1.060
                                  95%ci.ub    
tsens.(Intercept)                    0.850    
tsens.METHODOLOGY_CLEANMulti-omic    2.416    
tfpr.(Intercept)                    -1.355 ***
tfpr.METHODOLOGY_CLEANMulti-omic     0.099    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Variance components: between-studies Std. Dev and correlation matrix
      Std. Dev tsens  tfpr
tsens    0.890 1.000     .
tfpr     0.309 0.916 1.000

 logLik     AIC     BIC 
 17.611 -21.221 -15.813 


I2 estimates 
Zhou and Dendukuri approach:  49.3 % 
Holling sample size unadjusted approaches:  74.9 - 93.4 % 
Holling sample size adjusted approaches:  11.4 - 12.6 %

We do not seem to have enough evidence to conclude that the methodology used (panel vs. genome-wide) has a significant effect on the sensitivity and false positive rate. This may be due to a lack of enough studies in each subgroup, limiting the power of the analysis.

5.4 Academic vs. Commercial assays

An interesting question is whether the type of assay (academic vs. commercial) has an impact on the sensitivity and false positive rate of the ctDNA assays, although this is not a primary objective of the study, and interpretation should be done with caution.

5.4.1 Landmark setting

reitsma(
  data_landmark %>%
    filter(APPROACH == "Informed") %>%
    mutate(ACADEMIC_COMMERCIAL = factor(ACADEMIC_COMMERCIAL, levels = c("Academic", "Commercial"))),
  formula = cbind(tsens, tfpr) ~ ACADEMIC_COMMERCIAL
  ) %>%
  summary()

Call:  reitsma.default(data = data_landmark %>% filter(APPROACH == "Informed") %>% 
    mutate(ACADEMIC_COMMERCIAL = factor(ACADEMIC_COMMERCIAL, 
        levels = c("Academic", "Commercial"))), formula = cbind(tsens, 
    tfpr) ~ ACADEMIC_COMMERCIAL)

Bivariate diagnostic random-effects meta-analysis
Estimation method: REML

Fixed-effects coefficients
                                    Estimate Std. Error       z Pr(>|z|)
tsens.(Intercept)                     -0.076      0.125  -0.612    0.540
tsens.ACADEMIC_COMMERCIALCommercial    0.110      0.270   0.407    0.684
tfpr.(Intercept)                      -2.455      0.237 -10.336    0.000
tfpr.ACADEMIC_COMMERCIALCommercial    -0.231      0.502  -0.460    0.646
                                    95%ci.lb 95%ci.ub    
tsens.(Intercept)                     -0.321    0.168    
tsens.ACADEMIC_COMMERCIALCommercial   -0.420    0.640    
tfpr.(Intercept)                      -2.920   -1.989 ***
tfpr.ACADEMIC_COMMERCIALCommercial    -1.214    0.753    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Variance components: between-studies Std. Dev and correlation matrix
      Std. Dev tsens  tfpr
tsens    0.328 1.000     .
tfpr     0.786 0.755 1.000

 logLik     AIC     BIC 
 42.663 -71.327 -60.242 


I2 estimates 
Zhou and Dendukuri approach:  16 % 
Holling sample size unadjusted approaches:  23.2 - 48.4 % 
Holling sample size adjusted approaches:  0.9 - 0.9 %

reitsma(
  data_landmark %>%
    filter(APPROACH == "Agnostic") %>%
    mutate(ACADEMIC_COMMERCIAL = factor(ACADEMIC_COMMERCIAL, levels = c("Academic", "Commercial"))),
  formula = cbind(tsens, tfpr) ~ ACADEMIC_COMMERCIAL
  ) %>%
  summary()

Call:  reitsma.default(data = data_landmark %>% filter(APPROACH == "Agnostic") %>% 
    mutate(ACADEMIC_COMMERCIAL = factor(ACADEMIC_COMMERCIAL, 
        levels = c("Academic", "Commercial"))), formula = cbind(tsens, 
    tfpr) ~ ACADEMIC_COMMERCIAL)

Bivariate diagnostic random-effects meta-analysis
Estimation method: REML

Fixed-effects coefficients
                                    Estimate Std. Error      z Pr(>|z|)
tsens.(Intercept)                      0.120      0.394  0.305    0.760
tsens.ACADEMIC_COMMERCIALCommercial    0.356      0.511  0.698    0.485
tfpr.(Intercept)                      -1.931      0.195 -9.913    0.000
tfpr.ACADEMIC_COMMERCIALCommercial    -0.481      0.265 -1.814    0.070
                                    95%ci.lb 95%ci.ub    
tsens.(Intercept)                     -0.652    0.893    
tsens.ACADEMIC_COMMERCIALCommercial   -0.645    1.358    
tfpr.(Intercept)                      -2.312   -1.549 ***
tfpr.ACADEMIC_COMMERCIALCommercial    -1.000    0.039   .
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Variance components: between-studies Std. Dev and correlation matrix
      Std. Dev tsens  tfpr
tsens    0.612 1.000     .
tfpr     0.240 1.000 1.000

 logLik     AIC     BIC 
 21.751 -29.502 -23.269 


I2 estimates 
Zhou and Dendukuri approach:  0 % 
Holling sample size unadjusted approaches:  51.7 - 68 % 
Holling sample size adjusted approaches:  3.4 - 4.9 %

5.4.2 Serial setting

reitsma(
  data_serial %>%
    filter(APPROACH == "Informed") %>%
    mutate(ACADEMIC_COMMERCIAL = factor(ACADEMIC_COMMERCIAL, levels = c("Academic", "Commercial"))),
  formula = cbind(tsens, tfpr) ~ ACADEMIC_COMMERCIAL
  ) %>%
  summary()

Call:  reitsma.default(data = data_serial %>% filter(APPROACH == "Informed") %>% 
    mutate(ACADEMIC_COMMERCIAL = factor(ACADEMIC_COMMERCIAL, 
        levels = c("Academic", "Commercial"))), formula = cbind(tsens, 
    tfpr) ~ ACADEMIC_COMMERCIAL)

Bivariate diagnostic random-effects meta-analysis
Estimation method: REML

Fixed-effects coefficients
                                    Estimate Std. Error      z Pr(>|z|)
tsens.(Intercept)                      1.852      0.314  5.897    0.000
tsens.ACADEMIC_COMMERCIALCommercial   -0.058      0.552 -0.104    0.917
tfpr.(Intercept)                      -1.946      0.308 -6.308    0.000
tfpr.ACADEMIC_COMMERCIALCommercial    -1.931      0.777 -2.484    0.013
                                    95%ci.lb 95%ci.ub    
tsens.(Intercept)                      1.236    2.467 ***
tsens.ACADEMIC_COMMERCIALCommercial   -1.139    1.023    
tfpr.(Intercept)                      -2.551   -1.341 ***
tfpr.ACADEMIC_COMMERCIALCommercial    -3.454   -0.407   *
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Variance components: between-studies Std. Dev and correlation matrix
      Std. Dev tsens  tfpr
tsens    0.148 1.000     .
tfpr     0.581 1.000 1.000

 logLik     AIC     BIC 
 28.450 -42.901 -36.668 


I2 estimates 
Zhou and Dendukuri approach:  0 % 
Holling sample size unadjusted approaches:  0 - 0 % 
Holling sample size adjusted approaches:  0 - 0 %

reitsma(
  data_serial %>%
    filter(APPROACH == "Agnostic") %>%
    mutate(ACADEMIC_COMMERCIAL = factor(ACADEMIC_COMMERCIAL, levels = c("Academic", "Commercial"))),
  formula = cbind(tsens, tfpr) ~ ACADEMIC_COMMERCIAL
  ) %>%
  summary()

Call:  reitsma.default(data = data_serial %>% filter(APPROACH == "Agnostic") %>% 
    mutate(ACADEMIC_COMMERCIAL = factor(ACADEMIC_COMMERCIAL, 
        levels = c("Academic", "Commercial"))), formula = cbind(tsens, 
    tfpr) ~ ACADEMIC_COMMERCIAL)

Bivariate diagnostic random-effects meta-analysis
Estimation method: REML

Fixed-effects coefficients
                                    Estimate Std. Error      z Pr(>|z|)
tsens.(Intercept)                     -0.133      0.502 -0.265    0.791
tsens.ACADEMIC_COMMERCIALCommercial    1.047      0.698  1.499    0.134
tfpr.(Intercept)                      -1.726      0.189 -9.113    0.000
tfpr.ACADEMIC_COMMERCIALCommercial    -0.480      0.296 -1.625    0.104
                                    95%ci.lb 95%ci.ub    
tsens.(Intercept)                     -1.117    0.850    
tsens.ACADEMIC_COMMERCIALCommercial   -0.322    2.416    
tfpr.(Intercept)                      -2.097   -1.355 ***
tfpr.ACADEMIC_COMMERCIALCommercial    -1.060    0.099    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

Variance components: between-studies Std. Dev and correlation matrix
      Std. Dev tsens  tfpr
tsens    0.890 1.000     .
tfpr     0.309 0.916 1.000

 logLik     AIC     BIC 
 17.611 -21.221 -15.813 


I2 estimates 
Zhou and Dendukuri approach:  49.3 % 
Holling sample size unadjusted approaches:  74.9 - 93.4 % 
Holling sample size adjusted approaches:  11.4 - 12.6 %

6 Session info

sessionInfo()

R version 4.5.1 (2025-06-13)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 22.04.4 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.10.0 
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.0  LAPACK version 3.10.0

locale:
 [1] LC_CTYPE=es_ES.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=es_ES.UTF-8        LC_COLLATE=es_ES.UTF-8    
 [5] LC_MONETARY=es_ES.UTF-8    LC_MESSAGES=es_ES.UTF-8   
 [7] LC_PAPER=es_ES.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=es_ES.UTF-8 LC_IDENTIFICATION=C       

time zone: Europe/Madrid
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
 [1] mada_0.5.12         mvmeta_1.0.3        ellipse_0.5.0      
 [4] mvtnorm_1.3-3       metafor_4.8-0       numDeriv_2016.8-1.1
 [7] Matrix_1.7-4        meta_8.2-1          metadat_1.4-0      
[10] ggrepel_0.9.6       googlesheets4_1.1.2 devtools_2.4.6     
[13] usethis_3.2.1       cowplot_1.2.0       lubridate_1.9.4    
[16] forcats_1.0.1       stringr_1.6.0       dplyr_1.1.4        
[19] purrr_1.2.0         readr_2.1.6         tidyr_1.3.1        
[22] tibble_3.3.0        ggplot2_4.0.1       tidyverse_2.0.0    
[25] rmarkdown_2.30      knitr_1.50         

loaded via a namespace (and not attached):
 [1] tidyselect_1.2.1   farver_2.1.2       S7_0.2.1           fastmap_1.2.0     
 [5] CompQuadForm_1.4.4 mathjaxr_1.8-0     digest_0.6.39      timechange_0.3.0  
 [9] lifecycle_1.0.4    ellipsis_0.3.2     magrittr_2.0.4     compiler_4.5.1    
[13] rlang_1.1.6        tools_4.5.1        yaml_2.3.10        labeling_0.4.3    
[17] htmlwidgets_1.6.4  bit_4.6.0          pkgbuild_1.4.8     xml2_1.5.0        
[21] RColorBrewer_1.1-3 pkgload_1.4.1      withr_3.0.2        grid_4.5.1        
[25] googledrive_2.1.2  scales_1.4.0       MASS_7.3-65        cli_3.6.5         
[29] crayon_1.5.3       ragg_1.5.0         reformulas_0.4.2   generics_0.1.4    
[33] remotes_2.5.0      rstudioapi_0.17.1  tzdb_0.5.0         sessioninfo_1.2.3 
[37] minqa_1.2.8        cachem_1.1.0       splines_4.5.1      parallel_4.5.1    
[41] cellranger_1.1.0   vctrs_0.6.5        boot_1.3-32        jsonlite_2.0.0    
[45] hms_1.1.4          bit64_4.6.0-1      mixmeta_1.2.2      systemfonts_1.3.1 
[49] glue_1.8.0         nloptr_2.2.1       stringi_1.8.7      gtable_0.3.6      
[53] lme4_1.1-37        pillar_1.11.1      htmltools_0.5.8.1  R6_2.6.1          
[57] textshaping_1.0.4  Rdpack_2.6.4       vroom_1.6.6        evaluate_1.0.5    
[61] lattice_0.22-5     rbibutils_2.4      memoise_2.0.1      gargle_1.6.0      
[65] renv_1.1.4         Rcpp_1.1.0         nlme_3.1-168       mgcv_1.9-1        
[69] xfun_0.54          fs_1.6.6           pkgconfig_2.0.3